Multi-viewport display of multi-resolution hierarchical image

ABSTRACT

An input information obtaining portion of a control section obtains requests input from an input device by a user, which requests include a display region moving request to enlarge/reduce or scroll an image displayed on a display device and a request to generate/erase a viewport, change the size of a viewport, or move a viewport. A viewport control portion successively determines the number, arrangement, and size of viewports accordingly. A display region determining portion determines the region of an image to be displayed next in each viewport. A loading portion determines tile images to be newly loaded, and loads the data of the tile images from a hard disk drive. A decoding portion decodes the data of tile images used for rendering the image in each viewport. A display image processing portion updates the display region independently for each viewport.

TECHNICAL FIELD

The present invention relates to an image processing technology forchanging the display region of an image and displaying the image on adisplay.

BACKGROUND ART

An entertainment system for home use has been proposed which can notonly execute a game program but also reproduce a moving image. In theentertainment system for home use, a GPU generates three-dimensionalimages using polygons (see PTL 1, for example).

How to display an image efficiently is always an important problemirrespective of whether the image is a moving image or a still image.Therefore, various technologies have been developed and put to practicaluse in many fields, such as image data compression technology,transmission technology, image processing technology, displaytechnology, and the like. Thus, high-definition images have becomereadily available in various situations.

CITATION LIST Patent Literature

[PTL 1]

U.S. Pat. No. 6,563,999

SUMMARY Technical Problem

There is always a desire to display a high-definition image with goodresponsiveness according to a request by a user. In order to update adisplay image with good responsiveness in response to a viewpoint movingrequest with a degree of freedom, such as a request to enlarge anddisplay a region to which the user directs attention or move the displayto another region in a displayed entire image, for example, thetransfer, decoding, and rendering of image data in a random region needsto be completed within a limited time. Therefore, it is always a majorproblem to make image quality and responsiveness compatible with eachother within limits of processing performance and resources of a device.

The present invention has been made in view of such problems. It is anobject of the present invention to provide an image processingtechnology capable of displaying a high-definition image with goodresponsiveness in response to an operation related to a display region.

Solution to Problem

A mode of the present invention relates to an image processing device.The image processing device includes: an image data storage portion forstoring hierarchical data formed by layering, in order of resolution, aplurality of pieces of image data representing a display object imagewith different resolutions; a viewport control portion for determining alayer and a region used for rendering an image to be displayed in eachof a plurality of viewports generated on a screen of a display device,the layer and the region being included in the hierarchical data; adisplay region determining portion for independently updating the layerand the region used for rendering the image of a viewport of interestaccording to an input display region moving request signal; and adisplay image processing portion for reading data of the layer and theregion determined by the viewport control portion or the display regiondetermining portion from the image data storage portion, rendering theimage of each viewport using the data, and making the image of eachviewport displayed on the display device.

Another mode of the present invention relates to an image processingmethod. This image processing method includes: in an image processingdevice, a step of determining a layer and a region used for rendering animage to be displayed in each of a plurality of viewports generated on ascreen of a display device, the layer and the region being included inhierarchical data formed by layering, in order of resolution, aplurality of pieces of image data representing a display object imagewith different resolutions; a step of reading data of the determinedlayer and the determined region from a storage device, rendering theimage of each viewport using the data, and making the image of eachviewport displayed on the display device; a step of independentlyupdating the layer and the region used for rendering the image of aviewport of interest according to an input display region moving requestsignal; and a step of reading data of the layer and the region after theupdate from the storage device, and updating the image of the viewportof interest using the data.

It is to be noted that arbitrary combinations of the above constituentelements and modes realized by converting expressions of the presentinvention between a method, a device, a system, a computer program, andthe like are also effective as modes of the present invention.

Advantageous Effect of Invention

According to the present invention, it is possible to display an imagewith good responsiveness in response to an instruction by a user whilemaintaining desired image quality.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing a configuration of an image processingdevice according to a present embodiment.

FIG. 2 is a conceptual diagram of the hierarchical structure of imagedata used in the present embodiment.

FIG. 3 is a diagram schematically showing relations between the data oftile images used for rendering and a display screen in the presentembodiment.

FIG. 4 is a diagram mainly showing, in more detail, a configuration of acontrol section of the image processing device in the presentembodiment.

FIG. 5 is a diagram schematically showing parameters generated for aviewport control portion to control viewports in the present embodiment.

FIG. 6 is a flowchart of a processing procedure in which the imageprocessing device displays an image in the present embodiment.

FIG. 7 is a diagram showing an example of a display screen that can berealized by the device configuration of the present embodiment.

FIG. 8 is a diagram of assistance in explaining a method for making atotal area of viewports invariable in the present embodiment.

FIG. 9 is a diagram showing another screen example when a total area ofviewports is made invariable in the present embodiment.

FIG. 10 is a diagram of assistance in explaining a method for adjustinga layer used for rendering in the present embodiment.

FIG. 11 is a diagram of assistance in explaining the method foradjusting the layer used for rendering in the present embodiment.

FIG. 12 is a diagram showing another screen example when a layer usedfor rendering is adjusted in the present embodiment.

DESCRIPTION OF EMBODIMENTS

FIG. 1 shows a configuration of an image processing device according toa present embodiment. The image processing device 10 includes a wirelessinterface 40, an input device 20, a display processing section 44, adisplay device 12, a hard disk drive 50, a recording medium mountingsection 52, a disk drive 54, a main memory 60, a buffer memory 70, and acontrol section 100.

The display device 12 includes one of ordinary displays such as a liquidcrystal display, an EL (Electronic Luminescence) display, a plasmadisplay, and the like. The display device 12 may be provided integrallywith other modules of the image processing device 10, or may beconnected to the other modules of the image processing device 10 bywired connection or radio connection by a wire cable or a wireless LAN(Local Area Network), for example. The display processing section 44 hasa frame memory (not shown) for buffering data to be displayed on adisplay of the display device 12.

The wireless interface 40 is configured to be able to receive variouskinds of data such as image data and the like from a server by beingconnected to an external device or a network by radio according to apredetermined wireless communication protocol. The input device 20 isformed by ordinary input devices such as a joystick, a touch panel, amouse, a keyboard, buttons, and the like. The input device 20 at leastincludes input means for a user to input a display region moving requestto enlarge/reduce an image displayed on the display device 12 or scrollan image displayed on the display device 12 in an upward, downward,left, or right direction, a request to generate/erase a viewport to bedescribed later, change the size of a viewport, or move a viewport, andthe like.

Various concrete operating methods for performing these inputs areconceivable according to the type of the input device 20 and a method ofassigning functions to operating means included in the input device 20.For example, when a touch panel is used as the input device 20, a modecan be realized in which a finger in contact with a touch panel isshifted to thereby scroll a display image in a same direction, and thedisplay image is enlarged or reduced by opening or closing two fingers.An ordinary technology is applicable to processing related to suchoperations, and therefore description thereof will be omitted asappropriate. Various kinds of request signals input from the user on theinput device 20 are supplied to the control section 100.

The hard disk drive 50 functions as a storage device for storing data.The various kinds of data received from the server are stored on thehard disk drive 50. When a removable recording medium such as a memorycard or the like is mounted on the recording medium mounting section 52,the recording medium mounting section 52 reads data from the removablerecording medium. When a read-only ROM disk is mounted on the disk drive54, the disk drive 54 drives and recognizes the ROM disk, and reads datafrom the ROM disk. The ROM disk may be an optical disk, amagneto-optical disk, or the like. Various kinds of data such as imagedata and the like may be stored on these recording media.

The control section 100 includes a multi-core CPU. The control section100 has one general-purpose processor core and a plurality of simpleprocessor cores within the one CPU. The general-purpose processor coreis referred to as a PPU (PowerPC Processor Unit). The other processorcores are referred to as an SPU (Synergistic Processor Unit).

The control section 100 includes a memory controller connected to themain memory 60 and the buffer memory 70. The PPU has a register, andincludes a main processor as an entity of operation execution. The PPUefficiently assigns tasks as basic units of processing in applicationsbeing executed to the respective SPUs. Incidentally, the PPU itself mayexecute a task. The SPUs have a register, and include a sub processor asan entity of operation execution and a local memory as a local storageregion. The local memory may be used as the buffer memory 70.

The main memory 60 and the buffer memory 70 are storage devices, and areformed as RAMs (Random Access Memories). The SPUs have a dedicated DMA(Direct Memory Access) controller as a control unit, and enablehigh-speed data transfer between the main memory 60 and the buffermemory 70 and high-speed data transfer between the frame memory in thedisplay processing section 44 and the buffer memory 70. The controlsection 100 according to the present embodiment implements a high-speedimage processing function by operating a plurality of SPUs in parallelwith each other. The display processing section 44 is connected to thedisplay device 12, and outputs a result of image processing according toa request from the user.

The image processing device 10 according to the present embodiment loadsa part of compressed image data from the hard disk drive 50 into themain memory 60 in advance in order to change a display image smoothlywhen performing display image enlarging/reducing processing andscrolling processing. In addition, a part of the compressed image dataloaded in the main memory 60 is further decoded and stored in the buffermemory 70 in advance. This enables an image used in the generation ofthe display image to be changed instantly in subsequent necessarytiming.

The data structure of the image set as a processing object in thepresent embodiment is not particularly limited. However, description inthe following will be made by taking image data having a hierarchicalstructure as an example. The image data has a hierarchical structureincluding images of different resolutions which images are generated byreducing an original image in a plurality of stages. The image in eachlayer is divided into one or a plurality of tile images. For example, animage of a lowest resolution is formed by one tile image, and theoriginal image of a highest resolution is formed by a largest number oftile images. At a time of image display, tile images used for renderingare changed to tile images in a different layer when the display imageis changed to a predetermined resolution, and thereby enlarged displayor reduced display is made quickly.

FIG. 2 is a conceptual diagram of the hierarchical structure of imagedata. The image data has a hierarchical structure including a zerothlayer 30, a first layer 32, a second layer 34, and a third layer 36 in adirection of depth (Z-axis). Incidentally, while only the four layersare shown in FIG. 2, the number of layers is not limited to this. Theimage data having such a hierarchical structure will hereinafter bereferred to as “hierarchical data.”

The hierarchical data shown in FIG. 2 has a quadtree hierarchicalstructure. Each layer is formed by one or more tile images 38. All ofthe tile images 38 are formed in an identical size having a same numberof pixels, and for example have 256×256 pixels. Image data in each layerrepresents one image with a different resolution. Image data in thesecond layer 34, the first layer 32, and the zeroth layer 30 isgenerated by reducing the original image having the highest resolutionin the third layer 36 in a plurality of stages. For example, theresolution of an Nth layer (N is an integer of zero or more) may be ½ ofthe resolution of an (N+1)th layer in both of a left-right (X-axis)direction and an upward-downward (Y-axis) direction.

In the image processing device 10, the hierarchical data is retained inthe form of individual tile images 38 in the storage device in a stateof being compressed in a predetermined compression format, and is readfrom the storage device and decoded before being displayed on thedisplay. The image processing device 10 according to the presentembodiment has a decoding function supporting a plurality of kinds ofcompression formats, and is capable of decoding compressed data in aJPEG format, a JPEG2000 format, a PNG format, and an S3TC format.

In the hierarchical structure of the hierarchical data, as shown in FIG.2, a virtual three-dimensional space is constructed with the left-rightdirection set as an X-axis, the upward-downward direction set as aY-axis, and the depth direction set as a Z-axis. After the controlsection 100 derives an amount of movement of a display region on thebasis of a display region moving request signal supplied from the inputdevice 20, the control section 100 derives the coordinates of fourcorners of a frame (frame coordinates) in the virtual space using theamount of movement. The frame coordinates in the virtual space are usedto load compressed data into the main memory 60 and generate a displayimage.

For example, for changes in the resolution of the display image, changeboundaries are set between the layers on the Z-axis. When theZ-coordinate of a frame crosses such a change boundary according to adisplay region moving request signal, a layer used for rendering thedisplay image is changed. Then, loading and decoding are performed asrequired, and enlargement/reduction is performed according to arequested resolution, whereby the display image is generated.Incidentally, in place of the frame coordinates in the virtual space,the image processing device 10 may derive information identifying alayer and texture coordinates (UV coordinates) in the layer.Hereinafter, a combination of the layer identifying information and thetexture coordinates will also be referred to as frame coordinates.

The present embodiment realizes not only a mode in which one image isdisplayed on the entire screen but also a mode in which a plurality ofimages operable independently are displayed simultaneously. A sectionfor displaying an image within the screen will hereinafter be referredto as a “viewport,” irrespective of whether there is one image or aplurality of images. FIG. 3 schematically shows relations between thedisplay screen and the data of tile images used for rendering whenhierarchical data is used as image data.

In the present example, landscape image data 220 and superimposing imagedata 222 are prepared, the landscape image data 220 being obtained bylayering an image of a landscape photograph into a zeroth layer 220 a, afirst layer 220 b, a second layer 220 c, and a third layer 220 d, andthe superimposing image data 222 being obtained by layering an imageincluding character information desired to be displayed in a state ofbeing superimposed on the landscape photograph into a zeroth layer 222a, a first layer 222 b, and a second layer 222 c. Examples of the screenformed by two viewports using these pieces of hierarchical data arescreens 210, 212, and 214.

First, the screen 210 includes a first viewport 210 a in which the wholeof the landscape photograph is displayed on the entire screen and asecond viewport 210 b in which a part of the character information isdisplayed in a state of being superimposed at an upper right of thescreen. The image of the first viewport 210 a is rendered using the dataof tile images in the first layer 220 b of the landscape image data 220.The image of the second viewport 210 b is rendered using the data of apart of tile images in the first layer 222 b of the superimposing imagedata 222. The layers used in this case are determined by optimumresolution values set according to the sizes of the respective viewportsand the sizes of the display regions on the images.

The screen 212 includes a first viewport 212 a in which the whole of thelandscape photograph is displayed on the entire screen and a secondviewport 212 b in which a part of the landscape photograph is displayedin an enlarged and superimposed state. As in the case of the screen 210,the image of the first viewport 212 a is rendered using the data of thetile images in the first layer 220 b of the landscape image data 220.The image of the second viewport 212 b is rendered using the data of apart of tile images in the second layer 220 c of a higher resolution inthe same landscape image data 220.

The screen 214 includes a first viewport 214 a in which a part of thelandscape photograph is further enlarged than on the screen 212 anddisplayed on the entire screen and a second viewport 214 b in whichcharacter information in a wider range than on the screen 210 isenlarged and displayed in a state of being superimposed at an upper leftof the screen. The image of the first viewport 214 a is rendered usingthe data of a part of tile images in the third layer 220 d of thehighest resolution in the landscape image data 220. The image of thesecond viewport 214 b is rendered using the data of a part of tileimages in the second layer 222 c of a high resolution in thesuperimposing image data 222.

Thus, in the present embodiment, one or a plurality of viewports can bedisplayed, and images are rendered independently of each other inviewport units. A display region moving request for enlargement,reduction, or scrolling is received for each viewport. A request togenerate or erase a viewport or change the disposition or size of aviewport is also received. For example, when the user performs aninstruction input for specifying a partial region 211 within the firstviewport 210 a on the screen 210 and enlarging the partial region 211,the second viewport 212 b displaying the region is generated. Then, thescreen 212 can be displayed by increasing the size of the secondviewport 212 b according to an amount of enlargement.

Thus providing a plurality of viewports can realize flexible and freeimage expression. When the image data is hierarchical data, inparticular, even in the case of same images, a plurality of images thatare greatly differ from each other in resolution can be displayedsimultaneously as in the case of the screen 212. In addition, even inthe case of different kinds of images assumed to be synthesized witheach other such as a landscape image and a superimposing image or thelike, when these images are separate pieces of hierarchical data,display mode variations can be increased greatly as compared withpreparing originally synthesized image data. Even when there are aplurality of pieces of hierarchical data and a plurality of viewports, abasic processing procedure is not changed in terms of processing in tileimage units, so that data control is facilitated.

FIG. 4 mainly shows, in more detail, the configuration of the controlsection 100 of the image processing device 10 in the present embodiment.The control section 100 includes: an input information obtaining portion102 for obtaining information input by the user from the input device20; a viewport control portion 104 for controlling viewports accordingto the input by the user; a display region determining portion 106 fordetermining the region of an image to be displayed next in eachviewport; and a loading portion 108 for determining tile images to benewly loaded and loading compressed data from the hard disk drive 50.The control section 100 further includes: a decoding portion 110 fordecoding the compressed data; and a display image processing portion 114for rendering a display image.

The elements described as functional blocks performing variousprocessing in FIG. 4 can be formed by a CPU (Central Processing Unit), amemory, and another LSI in terms of hardware, and are realized by aprogram loaded in a memory or the like in terms of software. As alreadydescribed, the control section 100 has one PPU and a plurality of SPUs.The PPU and SPUs can form respective functional blocks singly or incooperation with each other. Hence, it is to be understood by thoseskilled in the art that these functional blocks can be implemented invarious forms by only hardware, only software, or combinations ofhardware and software, and are not to be limited to any one of theforms.

The input information obtaining portion 102 obtains details of aninstruction to start/end image display, select an image file, move adisplay region, or operate a viewport, for example, the instructionbeing input to the input device 20 by the user. The viewport controlportion 104 successively determines the number, disposition, and size ofviewports forming the screen according to user input informationnotified from the input information obtaining portion 102. For example,when an instruction input for specifying and enlarging the partialregion 211 on the screen 210 in FIG. 3 is performed, a new viewportshowing an enlarged image of the region is generated and disposed so asto be superimposed on a corresponding region in the original image, andthe size of the viewport at each time is determined according toenlargement speed. In addition to thus changing a viewport according toan instruction input from the user, the viewport control portion 104 maydetermine the disposition and size of a viewport according to settingsprepared together with image data.

Further, the viewport control portion 104 identifies a viewport as anobject of a display region moving request notified from the inputinformation obtaining portion 102 on the basis of the disposition of theviewport so that processing in subsequent stages which processing isinvolved in a display image update can be performed independently foreach viewport. Specifically, details of an operation such asenlargement, reduction, scrolling, or the like and the identifyinginformation of a viewport as an object of the operation are associatedwith each other, and notified to the display region determining portion106. The display region determining portion 106, the loading portion108, the decoding portion 110, and the display image processing portion114 perform their respective processing in association with theidentifying information of the viewport, and thereby display imagecontrol for each viewport can be realized.

The display region determining portion 106 determines frame coordinatesat each time determined by a frame rate, according to information on theframe coordinates of a present display region in each viewport and thedisplay region moving request input by the user. In addition, when framecoordinates change as the size of a viewport is changed, information onthe frame coordinates at each time is obtained from the viewport controlportion 104. The information on the frame coordinates is notified toeach of the loading portion 108, the decoding portion 110, and thedisplay image processing portion 114.

The loading portion 108 determines tile images to be loaded into themain memory 60 in advance on the basis of the information from thedisplay region determining portion 106. In addition, when there are tileimages yet to be loaded, the loading portion 108 loads the data of thetile images from the hard disk drive 50 into the main memory 60. Thetile images to be loaded into the main memory 60 in advance are not onlytile images included in the display region notified from the displayregion determining portion 106 but also tile images within apredetermined range around the periphery of the tile images included inthe display region, tile images in a region predicted to be necessaryhereafter from changes in frame coordinates in the past, and the like.The data of tile images other than those necessary in rendering animmediately subsequent display image may be loaded periodically atpredetermined time intervals, for example.

The same figure schematically shows a state in which the data of aplurality of tile image groups 62 is loaded in the main memory 60.However, loading processing may be performed in tile image units or inimage block units each formed by collecting together a plurality of tileimages. When index data (not shown) associating the identifyinginformation of the data of each tile image with the originalhierarchical data stored on the hard disk drive 50, a layer in thehierarchical data, and a position on the image is prepared separately,the data of tile images necessary for rendering can be identified on thebasis of frame coordinates. Hence, order of storage in the main memory60 is not particularly limited. In addition, all of image data can behandled uniformly in tile image units irrespective of the number ofpieces of hierarchical data and viewports.

The decoding portion 110 reads the data of necessary tile images fromthe main memory 60 on the basis of the information on the framecoordinates of each viewport which information is obtained from thedisplay region determining portion 106, decodes the data, and stores thedata in the buffer memory 70. The same figure schematically shows thedata 72 a and 72 b of two display images corresponding to two viewportsas the data stored in the buffer memory. In practice, however, as in themain memory 60, the order of storage of the data is not particularlylimited as long as the data is associated with the viewports to bedisplayed.

In addition, the number of pieces of corresponding display image datadiffers according to the number of viewports. The display image data 72a and 72 b is formed at least by the data of one or a plurality of tileimages including images of display regions in the respective viewports.Preferably, more tile images around the periphery of the display regionin each viewport are decoded, whereby the display image can be updatedsmoothly in response to a scrolling request by the user. In addition,when tile images in other layers representing the same region aredecoded, the display image can be updated smoothly also in response toan enlarging or reducing request.

The display image processing portion 114 includes an arranging block 120and a rendering block 122. The arranging block 120 associates an updateregion on the screen with the data of tile images necessary for updatingthe update region on the basis of the disposition and size of eachviewport on the screen which disposition and size are notified from theviewport control portion 104 and the frame coordinates notified from thedisplay region determining portion 106. According to the association,the rendering block 122 reads the data of the tile images from thebuffer memory 70, and renders a display image in the frame memory of thedisplay processing section 44 so as to update the display image withinthe viewport as the updating object on the basis of the notified framecoordinates.

FIG. 5 schematically shows parameters generated for the viewport controlportion 104 to control viewports. A rectangle 150 in FIG. 5 representsthe display screen of the display device 12. Suppose that theidentification number of a viewport corresponding to the screen is “0,”and that the identification numbers of two viewports 152 and 154displayed so as to be superimposed on the viewport corresponding to thescreen are “1” and “2,” respectively. The viewport control portion 104manages the viewports by the identification number of each viewport andthe disposition and size, for example the upper left coordinates andvertical/horizontal length of each viewport.

For example, as shown in FIG. 5, supposing that the upper leftcoordinates of the viewport (rectangle 150) having the identificationnumber “0” are (0, 0) and the horizontal and vertical lengths of theviewport having the identification number “0” are w0 and h0, theviewport is generated by creating a data set (0, (0, 0), (w0, h0)).Similarly, parameters (1, (x1, y1), (w1, h1)) are generated for theviewport 152, and parameters (2, (x2, y2), (w2, h2)) are generated forthe viewport 154. When a need to change the disposition and size of aviewport arises according to an operation by the user or the like, thecorresponding parameters other than the identifying information areupdated.

The viewport control portion 104 further associates the identificationnumber of each viewport with the frame coordinates of an image to bedisplayed first in the viewport. The example of FIG. 5 shows a state inwhich an image 156 and an image 160 are prepared as display objects, andin which the frame coordinates of the entire region of the image 156,the frame coordinates of a partial region 158 of the image 156, and theframe coordinates of a partial region 162 of the image 160 areassociated with the viewports having the identification numbers “0,”“1,” and “2,” respectively. While the size of a viewport is changingaccording to a viewport operation by the user or the like, the viewportcontrol portion 104 increases or decreases the range of the displayregion according to the size change, and determines the framecoordinates at each time.

Description will next be made of basic operation of the image processingdevice 10. FIG. 6 is a flowchart of a processing procedure in which theimage processing device 10 displays an image. First, an initial image isdisplayed according to an instruction input for selecting image data,starting display, or the like by the user (S10). This processing isperformed by loading, decoding, and rendering the data of an image setas the initial image. When the user performs an operation of changingthe region of a viewport by for example generating a new viewport orchanging the disposition or size of an originally present viewport (Y inS12), the viewport control portion 104 calculates the disposition andsize of the viewport according to the operation (S14).

Further, the viewport control portion 104 calculates the framecoordinates of a display region to be displayed in the viewport (S16).When the viewport is changed gradually, the processing of S14 and S16 isperformed for each time. Information on the disposition and size of theviewport is notified to the display image processing portion 114 inassociation with the identification number of the viewport. Informationon the frame coordinates is notified to the display region determiningportion 106 in association with the identification number of theviewport. Therefore, using the data of tile images loaded and decoded bythe loading portion 108 and the decoding portion 110, the display imageprocessing portion 114 disposes the viewport in an appropriate positionand updates the display image (S22).

On the other hand, when the operation of changing the region of theviewport is not performed, but an operation of moving the display regionwithin the viewport is performed (N in S12 and Y in S18), the viewportcontrol portion 104 notifies a corresponding display region movingrequest signal to the display region determining portion 106 as it is inassociation with the identification number of the viewport of interest.Receiving the display region moving request signal, the display regiondetermining portion 106 calculates new frame coordinates by adding anamount of movement according to the display region moving request signalto the frame coordinates thus far of the viewport of interest (S20).

Then, the display image of the viewport of interest is updated bycooperation of the loading portion 108, the decoding portion 110, andthe display image processing portion 114 (S22). While the user does notperform an instruction input for ending the image display (N in S24),the processing of S12 to S22 is repeated. When the instruction input forending the image display is performed, the processing is ended (Y inS24). Incidentally, the input device 20 is provided with operating meansthat determines the branch determinations of S12 and S18, that is,whether to change the region of a viewport or whether to move thedisplay region within the viewport. Alternatively, the determinationsmay be made by providing a rule to procedures of operation on the touchpanel and the like, and comparing the rule with an actually performedoperating procedure.

FIG. 7 shows an example of a display screen that can be realized by theabove-described device configuration. The example of the same figureassumes an electronic book of a travel guide. A screen 230 displays animage looking down at one page of a table of contents of the electronicbook. At this time, there is one first viewport formed of the entirescreen. When the user specifies a region 232 of an item of interest onthe screen 230, and performs an enlarging operation, the viewportcontrol portion 104 generates a new second viewport for displaying anenlarged image of the specified region. Then, the area of the secondviewport is increased according to the enlarging operation, and theframe coordinates are changed in an enlarging direction. The dispositionof the second viewport may be determined by the user by moving thesecond viewport, or may be determined according to a rule set inadvance.

Accordingly, the loading portion 108 loads the data of tile images in alayer changed as required, and the decoding portion 110 decodes thedata. The display image processing portion 114 uses the data to renderan image of a display region within the second viewport. A screen 234showing the second viewport 236 in a state of being superimposed on thefirst viewport is thereby displayed. When the image displayed in thesecond viewport 236 is rendered using data in a layer of a higherresolution than the image of the first viewport, a clearer enlargedimage can be displayed.

As for the specification of the region 232 on the screen 230, one ofregions set for each item in the original image in advance may beselected by using the input device 20, or the user may be allowed tospecify a region freely using the pointing device, the touch panel, orthe like forming the input device 20. The present embodiment can enlargeand reduce an arbitrary region in the display image by changing a layerused for rendering, and can therefore deal with free regionspecifications as in the latter case.

On the screen 234, the image of the table of contents which image isdisplayed in the first viewport is displayed in a state of being shiftedto the right as compared with the screen 230. Such scrolling may beperformed by the user, or a rule that the image be shifted in adirection of reducing an overlap region when the disposition of thesecond viewport 236 is determined, for example, may be set in advance.In this case, the viewport control portion 104 determines new framecoordinates of the display image in the first viewport, and notifies thenew frame coordinates to the display region determining portion 106.

When a region 237 displaying a photograph in the second viewport 236 onthe screen 234 is specified, and further an enlarging operation isperformed, the viewport control portion 104 for example returns theviewports to one viewport, and makes a screen 240 displayed which screen240 is formed by only an enlarged image of the specified region. Forexample, the size of the viewport itself is increased according to theenlarging request, and when the size of the viewport becomes identicalwith the size of the screen, the originally present first viewport iserased. Alternatively, a threshold value may be provided for anenlargement ratio, and when the enlargement ratio exceeds thepredetermined threshold value according to the enlarging request, theenlarged image may be displayed on the entire screen.

At this time, the enlarged image of the specified region may be renderedusing the data of a layer of an even higher resolution in thehierarchical data of the original image of the table of contents, ordisplay may be changed to an image of a page corresponding to thespecified item among text pages of the electronic book. In the lattercase, the viewport control portion 104 notifies the identifyinginformation of the hierarchical data of the image after the change tothe display region determining portion 106 together with framecoordinates. When the loading portion 108 accordingly loads the tileimages of the hierarchical data from the hard disk drive 50, subsequentprocessing is performed similarly irrespective of the hierarchical data.

On the other hand, when another region 238 in the first viewport on thescreen 234 is specified and an enlarging operation is performed, theviewport control portion 104 erases the second viewport 236 displayed onthe screen 234, and generates a third viewport 244 displaying anenlarged image of the specified region. As a result, a screen 242showing the third viewport 244 displaying the new enlarged image in astate of being superimposed on the first viewport is displayed.Alternatively, without the second viewport 236 on the screen 234 beingerased, the display image of the second viewport 236 may be updated tothe image of the newly specified region 238.

The example of FIG. 7 is an example of screen changes realized foroperations of enlarging a specified region. However, the presentembodiment can perform enlargement, reduction, and scrolling in eachviewport independently, as described above. Thus, screens that can bedisplayed are not limited to these screens. For example, the image inthe second viewport 236 can also be scrolled to display an adjacentitem.

In addition, assignment is performed to the input means of the inputdevice 20 as appropriate so that the second and third viewportsdisplayed on the screen 234 and the screen 242 can be erased, and sothat a new viewport displaying the original image of the table ofcontents can be displayed on the screen 240. These configurations enablefree image expressions such as changing the enlargement ratio of a sameimage in different regions or displaying different images in a state ofbeing superimposed on each other, and can realize contents that allowdesired information to be reached easily.

In the example shown in FIG. 7, various image expressions are performedby superimposing a plurality of viewports on each other on the screen.In this case, the larger the number and area of viewports, the largerthe number of tile images necessary for rendering. When the number oftile images as processing objects is increased, the load of data loadingprocessing and decoding processing is increased, and necessary resourcessuch as a bandwidth required for data transfer, data storage regions inthe main memory 60 and the buffer memory 70, and the like are increased.

On the other hand, when a display object image is prepared ashierarchical data, there is an advantage in that even when display ismade on any device, by appropriately selecting a layer as a processingobject according to the area of the display, the image can be displayedin a similar manner by only the processing of a number of tile imageswhich number is steady and suitable for the resources of each device.When the number and area of viewports are made variable as describedabove, an amount of necessary resources is not stable, so that the aboveadvantage of hierarchical data may be impaired in some cases.Accordingly, description will next be made of a method for realizing afree image expression using a plurality of viewports while maintainingthe advantage of hierarchical data. Incidentally, the method to bedescribed in the following can be combined with the modes described thusfar as appropriate.

FIG. 8 is a diagram of assistance in explaining a method for making atotal area of viewports invariable so that an amount of resourceconsumption is constant irrespective of the number of viewports. In thesame figure, as with the screen 240 in FIG. 7, for example, a screen 260is formed by only a first viewport 262 displaying an image obtained byenlarging the photograph included in the image of the table of contents.Specifically, the hierarchical data 270 of the image of the table ofcontents is set as a processing object, and the display image isrendered using the data of tile images of a partial region 272 in athird layer of the hierarchical data 270.

The screen 260 displays not only a first viewport 262 but also a tag 264shown in the shape of a band at a left edge of the screen. The tag 264is a GUI (Graphical User Interface) for generating a second viewport fordisplaying the whole of an image, or the whole of the image of the tableof contents in the example of FIG. 8. However, the shape and position ofthe tag 264 are not limited to this. When the user indicates the tag 264on the screen 260 via the input device 20, the tag 264 changes to asecond viewport 268, and the second viewport 268 is made to appeargradually according to an operation of scrolling in a right direction(screens 266 and 269). Preferably, when the input device 20 is the touchpanel on the screen, and the second viewport 268 is drawn out by afinger, an intuitive operation is realized.

The area of the second viewport 268 is increased by this display change.At this time, a total area of the first and second viewports is heldconstant by excluding a region in the display image of the firstviewport 262 which region is concealed by the overlapping of the secondviewport 268 from rendering objects. When the display image of thesecond viewport 268 is rendered in a first layer of the hierarchicaldata 270 as shown in FIG. 8, the second viewport 268 is rendered on thescreen 266 using the data of tile images in a right half region 274 ofthe image of the first layer which right half region 274 is shown on thescreen.

The second viewport 268 is rendered using the data of tile images in aregion 278 of the entire image of the first layer on the screen 269displaying the whole of the image of the table of contents. Meanwhile,as the first viewport 262 changes from the screen 260 to the screen 266to the screen 269, the concealed part of the first viewport 262 isexcluded from rendering objects, thereby decreasing the number of tileimages necessary for rendering the first viewport 262, as in the regions272, 276, and 280 in the third layer. As a result, a total number oftile images necessary for rendering the images of all of the viewportsis six in the state of any screen in the example of FIG. 8.

In actuality, the number of tile images as processing objects isslightly increased or decreased depending on positional relationsbetween the boundary lines of the display regions and divisions betweentile images as well as the number of tile images on the periphery of thedisplay regions which tile images should be decoded speculatively.However, even when such an increase or decrease is taken intoconsideration, an amount of necessary resources can be madesubstantially steady by thus excluding the concealed region fromrendering objects. For example, when the data of tile images renderedunnecessary is invalidated in the main memory 60 and the buffer memory70, the data of tile images rendered unnecessary can be overwritten withthe data of tile images for a new viewport. Thus, a necessary memorycapacity is not increased.

Incidentally, the two viewports are represented by using one piece ofhierarchical data in the example of FIG. 8. However, the display imagesof the respective viewports may be rendered using different pieces ofhierarchical data. In addition, display object image data may includenot only hierarchical data but also image data of one resolution.

In addition, as described above, each viewport is configured to allowthe display image therein to be controlled independently. Thus, forexample, on the screen 266, the display image in the first viewport 262can be enlarged, reduced, or scrolled, and the second viewport 268 canbe scrolled in an opposite direction to be retracted. In addition, whenthe region of one of items or photographs in the image of the table ofcontents is specified in the second viewport 268 on the screen 269, animage of the specified region may be displayed in an enlarged state inthe first viewport 262, or an image of a corresponding text page may bedisplayed in the first viewport 262.

FIG. 9 shows another screen example when a total area of viewports ismade invariable. An image of a page where text and photographs are mixedwith each other is assumed as a display object in the present example.First, a screen 282 is formed by one viewport displaying the whole ofthe page. When the user performs an input for specifying a text region284 on the screen 282, a second viewport 290 in which the text isenlarged so as to be viewed more easily is made to appear from thebottom of the screen, as shown on a screen 286. For example, when theinput device 20 is the touch panel on the screen, and the text region284 is touched for a predetermined time or more on the screen 282, anupper edge of the second viewport 290 appears at a lower edge of thescreen.

Then, the user can proceed with reading the text by scrolling the secondviewport 290 to an upper side (screen 288). An upper limit may be set tothe area of the second viewport 290, and after the upper limit isreached, only the text within the viewport may be scrolled by scrollingoperation without the area of the viewport being changed.

As shown on the left side of FIG. 9, as the image to be displayed in thesecond viewport 290, the data of an image 292 in which only the text isextracted may be prepared separately from an original page image 294. Inthis case, as indicated by arrows in the same figure, informationassociating regions of the image 292 of only the text with photographregions in the original page image 294 may be prepared on the basis ofcorrespondence relation between the text and the photographs in theoriginal page image 294, and as the text is scrolled in the secondviewport 290, a corresponding photograph may be displayed in the firstviewport. Incidentally, in the case of the page image in which the textregions and the photograph regions are originally separated from eachother in the form of columns, a similar mode can be realized by theidentical image data.

FIGS. 10 and 11 are diagrams of assistance in explaining a method foradjusting a layer used for rendering in order to make an amount ofresource consumption constant irrespective of the number of viewports.As with the screen 260 in FIG. 8, a screen 300 in FIG. 10 displays animage of an enlarged photograph in a first viewport 302 using the dataof tile images in a partial region 314 in a third layer of hierarchicaldata 310 of the image of the table of contents. A tag 304 is displayedat a left edge of the screen.

When the user indicates the tag 304 on the screen 300, and performs ascrolling operation of moving the tag 304 in a right direction, a secondviewport 308 displaying the whole of the image of the table of contentsappears in such a manner as to be drawn out from the position of the tag304, as shown on a screen 306. This is the same as in FIG. 8. However,in FIGS. 10 and 11, the display image of the first viewport 302 is leftas it is, and instead the layer used for rendering the second viewport308 is a layer of a lower resolution than a layer that should beoriginally used.

Supposing that an appropriate layer for displaying the whole of theimage in the size of the second viewport 308 is the first layer as shownin FIG. 8, the method of FIGS. 10 and 11 renders the second viewport 308using the data of a zeroth layer of a lower resolution than the firstlayer. The zeroth layer in the present example is formed by one tileimage 316. Thus, the number of tile images is not changed by changes inthe display region. The display image processing portion 114 enlargesthe tile image 316 to the size of the second viewport 308.

The first viewport 302 is rendered using the data of tile images in theregion 314 in the third layer, which is the same as used for renderingthe screen 300 in the hierarchical data 310. As shown in FIG. 10, aregion of the first viewport 302 which region coincides with the secondviewport 308 may be made visible by making translucent the whole of theimage of the second viewport 308 or only the background of the secondviewport 308 on the screen 306.

After the second viewport 308 continues being scrolled, the whole of theimage of the table of contents appears in the second viewport 308, asshown on a screen 312 in FIG. 11. Also at this time, the second viewport308 is rendered using the data of the tile image 316 in the zeroth layerof the hierarchical data 310, and the first viewport 302 is renderedusing the data of tile images in the region 314 in the third layer ofthe hierarchical data 310.

Then, at a point in time that the user stops the scrolling operation,the layer used for rendering the second viewport 308 is changed to theappropriate layer, and a region in the display image of the firstviewport 302 which region is concealed by the overlapping of the secondviewport 308 is excluded from rendering objects. A screen 320 displayedas a result is similar to the screen 269 in FIG. 8. That is, the firstviewport 302 is rendered using the data of tile images in a small region318 in the third layer, and the second viewport 308 is rendered usingthe data of tile images in a region 319 in the first layer.Incidentally, the change from the screen 312 to the screen 320 is notonly made when the whole of the image of the table of contents isdisplayed in the second viewport 308, but also it suffices for thechange to be made at a point in time that the user stops the scrollingoperation.

In general, attention is rarely paid to even details of an image beingscrolled, and attention is often paid to the image for the first time ata point in time that the scrolling is stopped. This tendency becomesstrong especially when scrolling operation is performed using the touchpanel on the screen in a state of the screen being hidden by a finger.That is, an image of a somewhat coarse image quality is considered tocause a little stress to the user in the middle of scrolling.

Utilizing this characteristic, for a viewport in the middle ofscrolling, a display image is generated by enlarging a tile image in alayer of a low resolution, and image data in an appropriate layer isused at a point in time that the scrolling is stopped. By thusestimating a degree of attention of the user and accordingly adjustingthe number of tile images by changing a layer used for rendering,variations in the number of tile images to be processed can besuppressed in any states of viewports, and an amount of necessaryresources can be made substantially steady.

In the example of FIGS. 10 and 11, the data in the layer of a lowresolution is used until the scrolling of the new second viewport 308made to appear is stopped. Also in a case where a display image withinan originally displayed viewport is scrolled, variations in the numberof tile images to be processed can be suppressed by performing similarprocessing. When the user performs scrolling at a high speed, forexample, it is necessary to predict a destination to which the displayregion moves hereafter and speculatively load and decode the data oftile images in a wider range, as described above, for a smooth imagetransition.

As a result, the number of tile images to be processed varies accordingto the scrolling speed. Accordingly, during a period during which thescrolling speed exceeds a predetermined threshold value, a layer usedfor rendering is changed to a layer of a low resolution, and the layerof the low resolution is displayed in an enlarged state. This does notpose a problem in the viewing of the image because there is a smallpossibility of the user closely checking the image being scrolled duringthe period of fast scrolling. As a result, it suffices to process abouta same number of tile images at all times, and variations in the amountof resource consumption can be suppressed.

FIG. 12 shows another screen example when a layer used for rendering isadjusted. This example shows various screens that can be displayed whenusing hierarchical data 350 of the image of the landscape photograph andhierarchical data 352 of the superimposing image including the characterinformation desired to be displayed in a state of being superimposed onthe landscape photograph. First, a screen 330 is formed by only a firstviewport 331 in which the whole of the landscape image is rendered usingtile images in a second layer 353 of a resolution suitable for the sizeof the display. When the user specifies a region 332 and inputs anenlarging instruction, a second viewport 336 displaying an enlargedimage of the region is generated, as shown on a screen 334.

The image of the second viewport 336 is rendered using the data of tileimages in a region 354 in a third layer having a resolution appropriatefor displaying the region in the size of the second viewport 336. Atthis time, the attention of the user is assumed to be focused on thesecond viewport 336. Thus, the layer used for rendering the firstviewport 331 is changed from the second layer 353 to a first layer 356to thereby decrease the number of tile images as a processing object.

Incidentally, the layer used for displaying the second viewport 336 maybe a layer of a lower resolution during a period during which the secondviewport 336 is enlarged in the process of a transition from the screen330 to the screen 334. This is because a degree of attention to theimage being enlarged is considered to be low especially when enlargingoperation is performed by a finger using the touch panel as in the caseof the above-described scrolling operation.

When an operation of enlarging the size of the second viewport 336 isperformed on the screen 334, and the range of the second viewport 336reaches the entire surface of the display, the original first viewport331 is erased, as shown on a screen 338. When a function of receivingthe operation of thus enlarging the size of a viewport is assigned to atouch panel provided separately on the back surface of the imageprocessing device 10, for example, the operation of thus enlarging thesize of a viewport can be separated from display image enlargingoperation on the touch panel on the front surface, that is, on thescreen. In this case, the display image is not hidden by a finger, andthus the second viewport 336 is rendered using the data in the thirdlayer of the same high resolution also in the process of the transition.

Incidentally, for a similar reason, the touch panel provided on the backsurface may be used for low-speed enlarging operation during which highimage quality is maintained, and the touch panel provided on the frontsurface may be used for high-speed enlarging operation with coarse imagequality. A third viewport 340 displaying a part of the superimposingimage is further made to appear on a screen 338 from a right edge of thescreen. This third viewport 340 may be displayed by scrolling operationby operating a tag as shown in FIG. 8 or the like, or such a functionmay be assigned to one of the input means of the input device 20.

The image of the second viewport 336 on the screen 338 is rendered usingthe data of tile images in a region 358 obtained by further expandingthe region 354 in the third layer used to display the screen 334 in thehierarchical data 350 of the landscape image. Meanwhile, the image ofthe third viewport 340 is rendered using the data of tile images in aregion 360 in a first layer, which has a lower resolution than a secondlayer having an appropriate resolution in the hierarchical data 352 ofthe superimposing image. This is because a degree of attention to theimage in the middle of scrolling is assumed to be low, as describedabove. The background of the image of the third viewport 340 at thistime is made transparent or translucent.

Then, when the operation of scrolling the third viewport 340 is stopped(screen 342), the data used for rendering the third viewport 340 ischanged to a region 364 in the second layer having the appropriateresolution, and a region of the second viewport 336 which region isconcealed by the third viewport 340 is excluded from rendering objects.As a result, tile images used for rendering in the hierarchical data 350of the landscape image are in a region 362 smaller than the region usedfor the screen 338. The screen 342 is in a similar state to the modeshown in FIG. 9. Thus, similar processing can be performed, such aschanging the image of the second viewport 336 to the image associatedwith the character information being displayed according to thescrolling of the character information within the third viewport 340,for example.

In addition to thus scrolling a viewport, also in cases where a viewportin which a specified region is enlarged is generated and where viewportsare generated by a plurality of pieces of hierarchical data,respectively, changing a layer being used as appropriate according to anestimated degree of attention or the like can suppress variations in thenumber of tile images as processing objects, and thus make an amount ofresource consumption steady.

According to the present embodiment described above, the number andarrangement of viewports forming a display screen are made variableaccording to operations by the user and rules set in advance, and framecoordinates are determined in association with the viewports, whereby adisplay region can be moved independently for each viewport. Therefore,in a case where a plurality of images such as a photograph and text orthe like are synthesized with each other and displayed, only one of theplurality of images can be scrolled, enlarged, or reduced, and thus freeimage expressions can be realized.

Here, when image data has a hierarchical structure formed by layeringdata of different resolutions, the enlargement and the reduction can beexpressed more dynamically, and also images that are generated from oneimage but greatly differ from each other in resolution can be displayedsimultaneously in different viewports. Further, by handling all of imagedata in tile image units divided in a predetermined size, loadingprocessing and decoding processing can be performed similarly even whena plurality of images are set as display objects, and complex imageexpressions can be realized even with a simple device configuration.

Further, in a situation in which a plurality of viewports are displayedin a state of being superimposed on each other, a part concealed by anoverlap is exclude from rendering objects. Alternatively, the image of aviewport to which a low degree of attention is predicted to be paid bythe user, such as a viewport in the middle of scrolling or the like, isrendered using image data of a low resolution. These measures enable anamount of data used for rendering to be held constant, and thus enablevariations in an amount of resource consumption to be suppressed, evenwhen the number and area of viewports are made variable. This iseffective particularly in a technology for realizing similar imagedisplay on various devices using versatile hierarchical data.

The present invention has been described above on the basis ofembodiments thereof. The foregoing embodiments are illustrative, and itis to be understood by those skilled in the art that combinations ofconstituent elements and processing processes of the embodiments aresusceptible of various modifications and that such modifications alsofall within the scope of the present invention.

REFERENCE SIGNS LIST

For example, in the present embodiment, description has been made of acase where mainly hierarchical data is used as a processing object.However, image data of one resolution may also be used. When such imagedata is divided into tile images and then processed as in the presentembodiment, for example, free image expressions can be realized bysimilar processing in tile image units irrespective of the number ofpieces of image data and the number of viewports. In addition, a similareffect of being able to make the number of tile images as processingobjects substantially constant and the like can be obtained bysubjecting viewports superimposed on each other to similar adjustmentsto those of the present embodiment.

An image as a processing object may be a still image or a moving imageregardless of whether or not the image is hierarchical data. An image asa processing object may be obtained by imaging a display screen of a webbrowser, for example. In such a case, an accurate and clear image can bedisplayed at all times even when free image operations such asenlargement, reduction, and the like are performed, without variouslanguages and font data in many sizes being prepared in a device.

10 Image processing device, 12 Display device, 20 Input device, 30Zeroth layer, 32 First layer, 34 Second layer, 36 Third layer, 44Display processing section, 50 Hard disk drive, 60 Main memory, 70Buffer memory, 100 Control section, 102 Input information obtainingportion, 104 Viewport control portion, 106 Display region determiningportion, 108 Loading portion, 110 Decoding portion, 114 Display imageprocessing portion, 120 Arranging block, 122 Rendering block.

INDUSTRIAL APPLICABILITY

As described above, the present invention is applicable to aninformation processing device such as a computer, an image processingdevice, an image display device, a portable terminal, a game machine,and the like.

The invention claimed is:
 1. An image processing device comprising: animage data storage portion for storing hierarchical data formed bylayering, in order of resolution, a plurality of pieces of image datarepresenting a display object image with different resolutions, each ofthe different resolutions being defined by a respective number of imagetiles; a viewport control portion for determining a layer and a regionused for rendering an image to be displayed in each of a plurality ofviewports generated on a screen of a display device, the layer and theregion being included in the hierarchical data; a display regiondetermining portion for independently updating the layer and the regionused for rendering the image to be displayed in each of the plurality ofviewports according to an input display region moving request signal,such that a substantially constant burden on the image processing deviceis maintained, irrespective of changes made in accordance with the inputdisplay region moving request signal, by maintaining substantiallyconstant a total number of the image tiles being used to render theimage; and a display image processing portion for reading data of thelayer and the region determined by the viewport control portion or thedisplay region determining portion from the image data storage portion,rendering the image of each of the plurality of viewports using thedata, and making the image of each of the plurality of viewportsdisplayed on the display device.
 2. The image processing deviceaccording to claim 1, wherein the viewport control portion changes asize of one or more of the plurality of viewports according to an inputviewport size changing request signal, and while changing the size ofthe one or more viewports, makes the layer used for rendering the imagedisplayed in the viewport a layer of a lower resolution than aresolution of an appropriate layer determined from the size of theviewport and size of the displayed image.
 3. The image processing deviceaccording to claim 2, wherein at a point in time that input of the inputviewport size changing request signal for increasing the size of the oneor more viewports is stopped, the viewport control portion changes thelayer used for rendering the image of the one or more viewports to theappropriate layer, and invalidates data within a memory, the data havingbeen used for rendering a region within another viewport, the regionwithin the other viewport being concealed by overlapping of the viewportwith the other viewport.
 4. The image processing device according toclaim 1, wherein the viewport control portion changes a size of one ormore of the plurality of viewports according to an input viewport sizechanging request signal, and invalidates data within a memory, the datahaving been used for rendering a region concealed by overlapping of theone or more viewports with another viewport among viewports constitutingthe screen, as an overlap area is increased.
 5. The image processingdevice according to claim 1, further comprising: an input section forreceiving an operation by a user for drawing out a tag displayed at anedge of the screen toward a center of the screen, wherein when thedrawing-out operation is started, the viewport control portion changesthe tag to a new viewport, and increases size of the viewport as theoperation proceeds.
 6. The image processing device according to claim 2,wherein while making the layer used for rendering the image displayed inthe viewport being changed in size the layer of the lower resolution,the viewport control portion makes a background of the image transparentor translucent.
 7. The image processing device according to claim 1,wherein according to an enlarging request signal specifying a regionwithin an image of a certain viewport, the viewport control portiongenerates a new viewport displaying an enlarged image of the region, andthe viewport control portion changes a display region of the image ofthe original viewport so as to decrease an area concealed by the newviewport.
 8. The image processing device according to claim 1, whereinaccording to a request signal to scroll an image of a certain viewport,the display region determining portion updates a region used forrendering the image such that the image is scrolled, and the displayregion determining portion changes an image of another viewport to animage associated with the display region after the scrolling.
 9. Animage processing method comprising: in an image processing device,storing hierarchical data formed by layering, in order of resolution, aplurality of pieces of image data representing a display object imagewith different resolutions, each of the different resolutions beingdefined by a respective number of image tiles; determining a layer and aregion used for rendering an image to be displayed in each of aplurality of viewports generated on a screen of a display device, thelayer and the region being included in hierarchical data; reading dataof the determined layer and the determined region from a storage device,rendering the image of each of the plurality of viewports using thedata, and making the image of each of the plurality of viewportsdisplayed on the display device; independently updating the layer andthe region used for rendering the image to be displayed in each of theplurality of viewports according to an input display region movingrequest signal, such that a substantially constant burden on the imageprocessing device is maintained, irrespective of changes made inaccordance with the input display region moving request signal, bymaintaining substantially constant a total number of the image tilesbeing used to render the image; and reading data of the layer and theregion after the update from the storage device, and updating the imageof each of the plurality of viewports using the data.
 10. Anon-transitory, computer readable storage medium containing a computerprogram for making a computer carry out actions, comprising: storinghierarchical data formed by layering, in order of resolution, aplurality of pieces of image data representing a display object imagewith different resolutions, each of the different resolutions beingdefined by a respective number of image tiles; determining a layer and aregion used for rendering an image to be displayed in each of aplurality of viewports generated on a screen of a display device, thelayer and the region being included in hierarchical data; reading dataof the determined layer and the determined region from a storage device,rendering the image of each of the plurality of viewports using thedata, and making the image of each of the plurality of viewportsdisplayed on the display device; independently updating the layer andthe region used for rendering the image to be displayed in each of theplurality of viewports according to an input display region movingrequest signal, such that a substantially constant burden on the imageprocessing device is maintained, irrespective of changes made inaccordance with the input display region moving request signal, bymaintaining substantially constant a total number of the image tilesbeing used to render the image; and reading data of the layer and theregion after the update from the storage device, and updating the imageof each of the plurality of viewports using the data.